Approved For Release 2006/05/23 : CIA-RDP84- ia fo Qi 73 le} 


ADMINISTRATIVE » INTERNAL USE ONLY 


MEMORANDUM FOR: Mr. W.E. Colby 


SUBJECT : Senior Seminar Procedures 


1. In looking over the material for Senior Seminar 
Three which is now underway, you inquired in writing about 
several procedures. One question concerned the ''Scope Notes" 
which describe the subject matter to be covered by guest” 
speakers in their presentations to the Seminar and you wondered 
if the speakers receive the Scope Notes. In arranging for a 
guest speaker, the Senior Seminar Staff drafts a tentative 
Scope Note which is forwarded to or discussed with the speaker. 
In many cases a Scope Note is rewritten as a result of these 
exchanges. This system has served the Senior Seminar quite 
well as the Scope Notes are incorporated into the schedule, | 
and Seminar members know at least in general terms what a 
speaker is expected to cover. Members are thus able to 
"bone-up" on the subject matter and direct their questions 
to better advantage. 


2. You also inquired whether good substantive ideas 
which are shared with the Seminar by non-Agency speakers, 
such as officers from State or the NSC Staff, are passed on 
to Agency management. The Senior Seminar Staff, whenever 
feasible, keeps a record of the material covered by guest 
speakers. In those instances when this material might be 
of interest to management, a Memorandum for the Record is 
produced and forwarded. An exampl nior Seminar One 


which was provided to Mr. Karamessines., © 
nother rec mple concerned the remarks by Mr. Andrew 


Marshall to the Seminar; of this MFR is attached. 


STAT 


Deputy Director for Support 
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MEMORANDUM FOR: Mr. W.E. Colby 


SUBJECT Senior Seminar Procedures 


1. In looking over tne material for Senior Seminar 
Three which is now underway, you inquired in writing about 
several procedures. One question concerned the "Scope Notes"! 
wnich descrive the subject matter to oe covered by guest 
speakers in their presentations to the Seminar and you wondered 
if the speakers receive the Scope Notes. In arranging for a 
guest speaker, the Senior Seminar Staff drafts a tentative 
Scope Note which is forwarded to or discussed with the speaker. 
In many cases a Scope Note is rewritten as a result of these 
excuanges. This system nas served the Senior Seminar quite 
well as the Scope Notes are incorporated into the schedule, 
and Seminar members know at least in general terms what a 
speaker is expected to cover. Members are thus able to 
"bone-up" on the subject matter and direct their questions 
to better advantage. 


2. You also inquired whether good substantive iceas 
which are shared with tae Seminar by non-Agency speakers, 
sucn as officers from State or the NSC Staff, are passed on 
to Agency management. The Senior Seminar Staff, whenever 
easible, keeps a record of the material covered by guest 
speakers, In those instances when this material might be 
of interest to management, a iemorandum for the Record is STAT 


produced and forwarded. An exampl Seminar One 
was tne remarks of Mr. Macomber on 
whica was provided to Mr. Karamessines. 


hotner recent example concerned the remarks by Nr. Andrew 
Marshall to the Seminar; a copy of this WFR is attacued. 


mee et ar ae ae a) a 
Fis) Maran opener ae ee ats 


jiJohn W. Coffey 
Deputy Director for Support 
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20 February 1975 


MEMORANDUM FOR THE RECORD: 


SUBJECT: Remarks of Mr. Andrew Marshall 


1. Mr. Marshall met with the Senior Seminar from 09830 
throuch 1900, Thursday, 15 February to discuss the topic 
“Intelligence Products: Problems and Improvenents.”" ivhen we 
considered inviting Mr. Marshall originally, it was with th 
idea of hearing hin talk on his work in the field of net 
assessments as a subject fittine the week's theme of (Xe oe 
Methodologies of Interest to Intelligence." Whea ot 
met with him, however, ‘fr. Marshall sugzested that he concen- 
trate instead on the products of intelligence to which we agreed 
in view of recent changes. Accordingly, while Mir. Marshall 
occasionally referred to new methods, it was in the context of 
exhortine CIA to exneriment with them, rather than the substance 
of net assessment as a new technique. 


2. Mr. Marshall began by saying that in his talk he would 
encourage us to enploy new techniques to a greater extent, but 
that he would also address the question of the difficulties which 
arise in trying to utilize them. He thon launched into a 
lengthy discussion of the intellirsence product as he has per- 
ceived it fron the vantass point of his earlier associations with 
the Agency and the Intellizence Community while at RAND and parti- 
cularly from the perspective of his present position as chief of 
the NSC Intelligence Conmmittec's Net Assessnent Groun. [fis _ 
remarks about the product were critical in varying degrees and 
are based, accordins to Nr. Marshall, on comments he has received 
from “consumers” whem he specifically identified as "members" of 
the NSC staff under Dr. Kissinger. 


3. His criticisms are as follows: 


a. The intelligence product and the production 
orsanizations are deficient in their "communication." In this 
connection he menticned writing style. Ue said we do not experi- 
ment enouch with different formats. He singled out several tines 
the lack of adequate summaries to accompany longer reports and 
critized our summaries. as "not hard hitting enough," i.e., they 
do not summarize the conclusions of the longer report well or 
firmly enough. He mentioned that “uncertainties”. concerning the 
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judgnents and estimates in the intelligence product are not. 
adequately srecified for the benefit of the reader. lie cited 
the "anonymity" of authorship as a major problem on the grounds 
that a consumer comes to trust an intellircence statement on the 
basis of knowing the individual whose judgement is expressed. 


b. The second set of criticisms concerned the "focus 
of the product."' This ‘ir. Marshall characterized as writing 
about the risht subjects in the risht way. Ife stated that he 
recognizes that intelligence procucers do not vet much feedback 
--cand that which is given is skewed in favor of praise. ile 
assigns blame to the consumer who is busy and does not know 
what he wants, but particularly to the producer. He criticized 
the Agency for failing to do "market research.’ He agreed that 
the questionnaires techniques is vrobably no good, but referred 
several times to the "Madison Avenue" approach. ie said that we 
should be more forceful and make a greater effort to fini out 
how we can help the consimer--again pointing to personnel contact 
between the area men in the NSC staff and area exnerts in the 
Agency. ‘With regard to focus, he also said he believes we should 
try more experimental products, the same products in a variety cf 
formats and particularly try to exnperinent with new nethodolosies. 


c. fr. Marshall then cited some "Oreanization and 
Management Changes" which he opined are. necessary in order to cet 
new approaches off the ground. He said that it has proven. most 
difficult to get experimental efforts geing because on-line: pro- 
duction offices are reluctant to put good people on problems 
beins dealt with in an experimental mode. He suggested that a 
separate unit(s) be set up for the sole purpose of experimentation 
in the field of intellisence production. lle cited trainins of 
analysts to work in the field of new methodologies as important. 


d. He then turned to the “quality of analysis." He 

- Stated that much of cur analysis is wanting in interpretation, 
not just occasional holes, but with respect to "whole programs." 
He charged that we fail to answer vital questions of "Why?" or 
“What Next?" and that we fail to put forth adequate hypotheses. 
He noted that there is not auch in the way of reasoning behind 
our intelligence statements and that they cannot be compared 

at a future date. In sum, he claimed, we have provided few 
"basic answers" over the 10 to 20 years of our existence. In 

a provocative assertion, he claimed that in order to provide 
answers to fundamental questions, he could do almost as well by 
dragging in a bunch of “people off the street." We just are not 
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good enough. As a corrective, he urged we adopt the use of 
footnotes citing sources and append annexes which lay out the 
reasoning behind intelligence statements. He repeated that it 
is difficult to find “real analysis" in our products. He also 
suggested that we try harder to reach "our real readers" who, 
he suggested, are the staff persons working for top leaders, 
e.g., the NSC staff. He again stressed the importance of cood 
summaries for top readers. He suggested that we experiment 
more with visual, rather than word, presentations of material, 
e.g., Cartoons. To improve the analytical quality of our pro- 
duct, he proposed that we institute an R&D effort in analysis, 
giving in-depth study to selected topics and that we make an 
effort to develon means of conducting a critical review on a 
sampling basis in order to get feedback. 


e. Turning to new methods, he held up Bayesian 
analysis as providing a systematic way of handling uncertainties, 
citing Dr. Kissincer' $s comment to him that policy-making is 
"betting on the future." “Mr. Marshall suggested, that we might 
try expressing our intelligence judgments to Dr. Kissinger in 
the form of betting odds. le also mentioned the Burcaucratic 
Politics model of analysis followed by Graham Alison as a useful 
means of interpreting the behavior of other governments. He 
cited Kuhn's Scientific Revolution as useful reading, and held 


up the behavioral sciences as potentially useful in gaining 


} 3 i sey g aders, noting the 
s helpful. 


3. As to "cautionary tales," Mr. Marshall gave his view 
that game theory has not been very useful. He cited the barriers 
that exist to innovation, giving as an example a rather lengthy 
effort on his part going back to 1958 while he was at RAND to 

get the Asency to take a new tack in analyzing Soviet weapons pro- 
grams. 


4, Mr. Marshall stated, in summary, that he feels that the 
Agency should take the leadership in certain key areas vis-a-vis 
universities and think-tanks. [He said in some cases these are 
areas in which CIA is falling behind. He urged that the Agency 
have closer contact with this outside work. Ie observed that we 
should not expect this work to be developed outside to.the extent 
that it then be handed to CIA and immediately applied. 
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5. The question and answer session served mainly to 
amplify some of tir. Marshall's statements. One exchange con- 
cerned the unwillingness of the consumer to "level" witha CIA 
--saying only good things to us, but voicing dissatisfaction 
to others such as Marshall. lle again implied that the burden 
of producer-consuner communication falls on the intelligence 


producer. 


eninar 
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MEMORANDUM FOR: Deputy Director for Support 


SUBJECT Status Report on Senior Seminar Three 


1, There is attached the roster of the officers 
picked by the Training Selection Board to attend Senior 
Seminar Three plus those designated as alternates. Also 
attacned is a set of draft schedules which reflect the 
present Status of planning for Seminar Three; items which 
are not yet firm are shown in brackets. In view of our 
discussion at your briefing by the OTR Curriculum Council 
regarding the problem of defining the subject matter to 
be covered by guest speakers, I am also attaching copies 
of the Scope Notes which have been developed thus far. 
These notes, which will be interleaved with the details 
on the draft schedules in the final scnedule, are some- 
wnat bulky but are forwarded in view of your interest. 


2. The major changes in Seminar Three include a week 
devoted to "New Methodologies of Interest to Intellicence.” 
This week (Block III) concentrates on quantitative method- 
Ologies and tools such as ADP and their usefulness to in- 
telligence analysis, collection and management of intelli- 
gence resources, Covert Action, which was treated separately 
in the first two runnings, has been integrated in Seminar 
Three into the two weeks (Block II) devoted to the Agency's 
intelligence activities. The subject matter of the con- 
Cluding Block, "Management of Intelligence in a Period of 
Cnange," has been restructured to help Seminar members tie 
together all facets of the nine weeks. This Block includes 
several presentations based on the premise that the best 
forn of management training comes from our senior managers 
themselves, Finally, in this running the individual pres- 
entations by Seminar members are being worked out with the 
participants by the Staff and scheduled in advance in an 
effort to further increase the value of these inputs wnich 


z 


have been such a successful feature of tha Senior Seminar. 


3. In designing and developing Seminar Three, the 
Staff nas been especially alert to ir. Colby's suggestions 
on Seminar Two in his memorandun of March 1972. In addition 
to the above mentioned emphasis on new methodologies, there 
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will be improved coverage of CIA-State relations with "A 
Day at the State Department"; more extensive coverage of 
the NSC and what is expected from CIA by that quarter (Hal 
Saunders, Andrew Marshall, Jeanne Davis and a briefing in 
the White House Situation Room); a closer look at the ana- 
lytical and production functions; a presentation on the 
European Community by Professor Bowie of Harvard, an ONE 
consultant; and increased use of films. 7 


4. In sum, I believe you will find the program out- 
lined in these schedules will be interesting and challenging 
to the Seminar participants and that their participation in 
Seminar Three will play a significant role in their develop- 
ment as senior officers. . 
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